Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Map model library locally #240

Merged
merged 2 commits into from
Dec 19, 2023
Merged

Map model library locally #240

merged 2 commits into from
Dec 19, 2023

Conversation

CharlieFRuan
Copy link
Contributor

Instead of relying on local_id and model_lib in the mlc-chat-config.json on huggingface, we map it internally within AppConfig by adding a field model_lib in each ModelRecord in model_list.

@CharlieFRuan
Copy link
Contributor Author

cc @tqchen

tqchen
tqchen reviewed Dec 18, 2023
View reviewed changes
README.md Outdated
@@ -131,6 +131,7 @@ async main() {
"model_list": [
{
"model_url": myLlamaUrl,
"model_lib": "llama-v1-3b-q4f32_0",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can use model_lib: to directly point to a wasm url, so remove the model_lib indirection

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That also enables us to remove the lib map indirection and focus on concept of "model" and "model library"

@CharlieFRuan
Copy link
Contributor Author

@tqchen Just removed model_lib_map. I am still keeping the interface AppConfig despite it now only having one required field model_list after removing the map; was thinking that it could be useful for cases like simple-chat's "use_web_worker". Please let me know if you think we should remove AppConfig altogether.

Besides, should we also rename local_id to perhaps model_name? Though the name is still used in mlc-llm.

@tqchen
Copy link
Contributor

tqchen commented Dec 19, 2023

Great, let us think about the name local_id and see what to do

@tqchen tqchen merged commit 490ddc1 into mlc-ai:main Dec 19, 2023
@CharlieFRuan CharlieFRuan mentioned this pull request Dec 26, 2023
Merged
@CharlieFRuan CharlieFRuan deleted the pr-1217-modellib-map branch February 15, 2024 20:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tqchen tqchen tqchen left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@CharlieFRuan @tqchen